Skip to content

Enable measures to return rich outputs (plots, HTML tables) - #113

Merged
simonpcouch merged 33 commits into
mainfrom
rich-output-56
Aug 20, 2026
Merged

Enable measures to return rich outputs (plots, HTML tables)#113
simonpcouch merged 33 commits into
mainfrom
rich-output-56

Conversation

@skaltman

@skaltman skaltman commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Addresses #56.

I mostly wanted to see what the TLG agent felt like with plots and gt tables supported as measure outputs, so I started working on this.

Totally fine if we want to go in a different direction for this.

Idea for more general solution from sol:

Future work: Consider generalizing rich_table() into a rich_output() abstraction that
separates user-facing HTML, model-facing semantic content, and a calculation-ready
run_r value. rich_table() could remain a table-specific wrapper, while the same
foundation could support maps, HTML widgets, and other interactive visualizations
without applying table-specific rendering or model instructions.

image image

@github-actions

github-actions Bot commented Aug 16, 2026

Copy link
Copy Markdown

Preview deployed to Connect (dogfood.team.pct.posit.it): https://dogfood.team.pct.posit.it/connect/#/apps/d7a36cae-8f27-448b-a478-61b81fbe3942/draft/362800

Deployed from commit a48ba1c.

@github-actions

github-actions Bot commented Aug 16, 2026

Copy link
Copy Markdown

Preview deployed to Connect (connect.staging.pct.posit.it): https://connect.staging.pct.posit.it/connect/#/apps/ad662e1b-5048-4acc-9ad7-f9478c92274e/draft/1996

Deployed from commit a48ba1c.

@simonpcouch
simonpcouch marked this pull request as ready for review August 18, 2026 21:00
@simonpcouch

Copy link
Copy Markdown
Collaborator

Thank you, @skaltman! I've made some changes here—the big item is removing the exported rich_table() helper in favor of automatically showing ggplots and gt tables when they're the tool result of a measure.

Could you install the new version of this PR and then take it for a spin in the TLG agent and let me know if it works for you? You might need to remove any calls to rich_table().

@skaltman

skaltman commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

@simonpcouch thank you! The plot support works great. The tlg-agent complex tables aren't actually gt_tbl objects, though, so those measures currently don't work as intended and instead fall back to the generic printed output, which is closed by default. The non-rectangular TLG Catalog tables are rtables::TableTree objects rendered as HTML with rtables::as_html(). (I realized that my PR description's mention of gt was a bit of a red herring.)

Since this is just an example app, I don't think it's that big of a deal to convert them to gt, but do you see a different way forward that provides more general support for HTML tables?

I realize that gt tables are nice because they carry both the HTML representation and the underlying data frames, so commons can run additional operations on them. Generic HTML support would need to do something different, which is why rich_table() coupled the HTML with a separate value.

@simonpcouch

simonpcouch commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

I guess the approach that would allow us to reuse as much of what ellmer/shinychat give us as possible, and also an entry point to model-facing vs. user-facing display, is that measures could return ellmer::ContentToolResults:

  table <- build_table(...)

  ellmer::ContentToolResult(
    value = rtables::as_result_df(table),
    extra = list(
      display = shinychat::tool_result_display(
        html = rtables::as_html(table),
        open = TRUE,
        show_request = FALSE
      )
        data = table
    )
  )

Then:

  • the model receives the result data frame;
  • Commons will register table for run_r
  • shinychat displays the authored rtables HTML.

@simonpcouch

Copy link
Copy Markdown
Collaborator

Ah, looks like tlg-agent depended on the old definitions syntax and my merge upstream broke it. Working on a fix for tlg-agent locally!

@simonpcouch

Copy link
Copy Markdown
Collaborator

Just gave this a go in tlg-agent and it feels good to me! If you could, please take a look + take that for a spin and, if it works well for you, feel free to squash+merge here!

@skaltman

skaltman commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

It works great in tlg-agent! Just waiting for the checks to complete...

@simonpcouch
simonpcouch merged commit f3110f1 into main Aug 20, 2026
1 of 4 checks passed
@simonpcouch
simonpcouch deleted the rich-output-56 branch August 20, 2026 13:53
@github-actions

Copy link
Copy Markdown

Cleaned up 14 preview bundle(s) on https://dogfood.team.pct.posit.it: 361854, 361856, 361857, 361859, 361860, 362424, 362425, 362426, 362429, 362432, 362433, 362795, 362799, 362800

@github-actions

Copy link
Copy Markdown

Cleaned up 14 preview bundle(s) on https://connect.staging.pct.posit.it: 1805, 1806, 1807, 1808, 1809, 1916, 1917, 1919, 1920, 1923, 1924, 1994, 1995, 1996

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants